Questions Are All You Need to Train a Dense Passage Retriever

Abstract We introduce ART, a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled data. Dense is central challenge open-domain tasks, such as Open QA, where state-of-the-art methods typically large supervised datasets with custom hard-negative mining and denoising of positive examples. in contrast, only requires access to unpaired inputs outputs (e.g., questions potential answer passages). It uses passage-retrieval scheme, (1) an input question used retrieve set evidence passages, (2) the passages are then compute probability reconstructing original question. Training based on reconstruction enables effective unsupervised learning both passage encoders, which can be later incorporated into complete QA systems without further finetuning. Extensive experiments demonstrate ART obtains results multiple benchmarks generic initialization from pre-trained language model, removing need data task-specific losses.1 Our code model checkpoints available at: https://github.com/DevSinghSachan/art.